1. Technical Field
The present invention relates to anomaly detection on packet switched communication systems. Particularly, the present invention is related to statistical methods for detecting network traffic anomalies due to network attacks or to communication system failures.
2. Description of the Related Art
Several types of attacks are known, such as: (distributed) denial of service ((D)DoS) attacks, scanning attacks, SPAM or SPIT attacks, and malicious software attacks.
Denial-of-Service (DoS) attacks and, in particular, distributed DoS (DDoS) attacks are commonly regarded as a major threat to the Internet. A DoS attack is an attack on a computer system or network that causes a loss of service or network connectivity to legitimate users, that is, unavailability of services. Most common DoS attacks aim at exhausting the computational resources, such as connection bandwidth, memory space, or CPU time, for example, by flooding a target network node by valid or invalid requests and/or messages. They can also cause disruption of network components or disruption of configuration information, such as routing information, or can aim at disabling an application making it unusable. In particular, the network components (e.g., servers, proxies, gateways, routers, switches, hubs, etc.) may be disrupted by malicious software attacks, for example, by exploiting buffer overflows or vulnerabilities of the underlying operating system or firmware.
A DDoS attack is a DoS attack that, instead of using a single computer as a base of attack, uses multiple compromised computers simultaneously, possibly a large or a very large number of them (e.g., millions), thus amplifying the effect. Altogether, they flood the network with an overwhelming number of packets which exhaust the network or application resources. In particular, the packets may be targeting one particular network node causing it to crash, reboot, or exhaust the computational resources. The compromised computers, which are called zombies, are typically infected by malicious software (worm, virus, or Trojan) in a preliminary stage of the attack, which involves scanning a large number of computers searching for those vulnerable. The attack itself is then launched at a later time, either automatically or by a direct action of the attacker.
(D)DoS attacks are especially dangerous for Voice over IP (VoIP) applications, e.g., based on the Session Initiation Protocol (SIP). In particular, the underlying SIP network dealing only with SIP signaling packets is potentially vulnerable to request or message flooding attacks, spoofed SIP messages, malformed SIP messages, and reflection DDoS attacks. Reflection DDoS attacks work, as an example, by generating fake SIP requests with a spoofed (i.e. simulated) source IP address, which falsely identify a victim node as the sender, and by sending or multicasting said SIP requests to a large number of SIP network nodes, which all respond to the victim node, and repeatedly so if they do not get a reply, hence achieving an amplification effect.
SPAM attacks consist in sending unsolicited electronic messages (e.g., through E-mail over the Internet), with commercial or other content, to numerous indiscriminate recipients. Analogously, SPIT (SPam over Internet Telephony) attacks consist in sending SPAM voice messages in VoIP networks. Malicious software attacks consist in sending malicious software, such as viruses, worms, Trojan, or spyware, to numerous indiscriminate recipients, frequently in a covert manner. Scanning or probing attacks over the Internet consist in sending request messages in large quantities to numerous indiscriminate recipients and to collect the information from the provoked response messages, particularly, in order to detect vulnerabilities to be used in subsequent attacks. For example, in port scanning attacks, the collected information consists of the port numbers used by the recipients.
Attack detection techniques are known which utilize a description (signature) of a particular attack (e.g., a virus, worm, or other malicious software) and decide if the observed traffic data is consistent with this description or not; the attack is declared in the case of detected consistency.
Furthermore, anomaly detection techniques are known which utilize a description (profile) of normal/standard traffic, rather than anomalous attack traffic, and decide if the observed traffic data is consistent with this description or not; an attack or anomalous traffic is declared in the case of detected inconsistency.
Unlike attack detection techniques, anomaly detection techniques do not require prior knowledge of particular attacks and as such are in principle capable of detecting previously unknown attacks. However, they typically have non-zero false-negative rates, in a sense that they can miss to declare an existing attack. They also typically have higher false-positive rates, in a sense that they can declare anomalous traffic in the case of absence of attacks.
Anomaly detection techniques can essentially be classified into two categories: rule-based techniques and statistic-based or statistical techniques. Rule-based techniques describe the normal behavior in terms of certain static rules or certain logic and can essentially be stateless or stateful. In particular, such rules can be derived from protocol specifications.
On the other hand, statistical anomaly detection techniques describe the normal behavior in terms of the probability distributions of certain variables, called statistics, depending on the chosen data features or parameters.
Paper “Characteristics of network traffic flow anomalies,” P. Barford and D. Plonka, Proceedings of the 1st ACM SIGCOMM Workshop on Internet Measurement, San Francisco, Calif., November 2001, pp. 69-73, suggests that packet rate, byte rate, and flow rate (i.e., the number of packets, bytes, and flows per second) curves in time can be useful for detecting and classifying traffic anomalies, possibly through the wavelet transform techniques.
US-A-2003/0200441 describes a method for detecting (D)DoS attacks based on randomly spoofed (i.e., simulated) IP addresses. To reduce the number of IP addresses, they are first hashed by a hash function. The method consists of counting the relative number of different values of hashed IP addresses among a number of packets, which are inspected successively in time, and of comparing this number with a predetermined threshold. A (D)DoS attack is declared if the threshold is exceeded. The number of inspected packets is iteratively increased if a (D)DoS attack is not detected.
Article “Proactively detecting distributed denial of service attacks using source IP address monitoring”, T. Peng, C. Leckie, and K. Ramamohanarao, Proceedings of Networking 2004, Lecture Notes in Computer Science, vol. 3042, pp. 771-782, 2004, discloses a method according to which DDoS attacks can be (proactively) detected even near the sources of the attack by checking for an increase of new source IP addresses appearing, provided that the source IP addresses of the attack traffic are randomly spoofed. It should be noticed that according to this article the IP addresses are monitored in non-overlapping time intervals and the increase is measured with respect to a database of legitimate IP addresses collected during off-line training.
Paper “Mining anomalies using traffic feature distributions”, A. Lakhina, M. Crovella, and C. Diot, Proceedings of SIGCOMM '05, Philadelphia, Pa., Aug. 22-26, 2005, pp. 217-228, discloses a method comprising a step of computing the “sample entropy” of discrete packet features such as IP addresses and port numbers, in non-overlapping, relatively short time intervals (e.g., 5 min), to statistically model the multidimensional entropy data collected on multiple links in a communications network by using the principal component analysis, and then to verify if the current data is inconsistent with the model determined by checking if the squared prediction error, resulting from the principal component analysis exceeds a threshold. The Applicant observes that the sample entropy used is based on the well-known Shannon entropy. It is expected that the frequency distribution of the IP addresses or port numbers reflected in the sample entropy should change in the case of an attack traffic. The Applicant observes that the same method is later proposed in WO-A-2007/002838.
Article “Entropy based worm and anomaly detection in fast IP networks”, A. Wagner and B. Plattner, Proc. 14. IEEE International Workshops on Enabling Technologies Infrastructure for Collaborative Enterprises, Linköping, Sweden, June 2005, pp. 172-177, discloses a method considering discrete packet features such as IP addresses in relatively short time intervals (e.g., 5 min) and to compress a concatenation of all the IP addresses occurring in the interval by a lossless data compression algorithm, such as the Lempel-Ziv coding algorithm. It is expected that the compression ratio should be lower if there is an attack traffic in the interval, due to randomization of destination IP addresses.
Thesis “DDoS attack detection based on Netflow logs”, E. Haraldsson, Student thesis SA-2003.35, Swiss Federal Institute of Technology, Zurich, 2003, and thesis “Plug-ins for DDoS attack detection in realtime,” A. Weisskopf, Semester thesis SA-2004.19, Swiss Federal Institute of Technology, Zurich, 2004, disclose a number of packet statistics for the detection of DDoS attacks. The statistics examined by Haraldsson include the number of open or half-open (obtained from TCP flags) connections, the number of transmitted or received bytes per (grouped) IP address, the number of open ports per (grouped) IP address, and the histogram of the average packet sizes, while the statistics examined by Weisskopf include the histogram of the flow sizes in bytes over a time period and the activity of (grouped) IP addresses.